Papers with sub-word tokenization
Unsupervised Term Extraction for Highly Technical Domains (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Term extraction is an important task for knowledge discovery platforms because domain specific terms are the linguistic representation of domainspecific concepts. |
| Approach: | They propose a term extraction subsystem that uses an unsupervised annotator to generate training data to fine-tune transformer models. |
| Outcome: | The proposed system can generalize across domains while reducing latency and inference time while preserving the high performance of the existing system. |
Low-resource neural machine translation with morphological modeling (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for character-based and sub-word tokenization are limited to the surface forms of the words. |
| Approach: | They propose a framework-solution for modeling complex morphology in low-resource settings using a transformer architecture and beam search-based decoder. |
| Outcome: | The proposed model improves translation performance on Kinyarwanda English translation using public-domain parallel text. |
Neural Machine Translation with Synchronous Latent Phrase Structure (2021.acl-srw)
Copied to clipboard
| Challenge: | grammatical information annotation requires high human resources and is not trivial due to language mismatches and out-of-vocabulary problem. |
| Approach: | They propose to incorporate grammatical information without supervising annotation by induced latent phrase structure and synchronized phrase structures in encoder and decoder to enhance explainability. |
| Outcome: | The proposed method produces better performance and explainability in translation and alignment tasks without extra resources. |
To Split or Not to Split: Composing Compounds in Contextual Vector Spaces (2023.emnlp-main)
Copied to clipboard
| Challenge: | Contextual word embedding models rely on sub-word tokenization to represent single orthographic words but are often suboptimal in under-resourced contexts. |
| Approach: | They propose to use a masked language modelling task to evaluate the model's performance . they use re-trained tokenizers to pre-split compounds into constituents . |
| Outcome: | The proposed models improve on the masked language modelling task and compositionality prediction by pre-splitting compounds into constituents. |